Provided are a training set and a test set of images of 3 classes. Each image has a filename that is its unique id. The dataset comprises 3 classes:
The goal of the project is to create a classifier capable of determining the class of the X-ray image.
The context is:
The project is inspired from a dataset from Kaggle. Note: For project purposes, the dataset is modified according to the learning requirements. You are provided with the following files:
The dataset (above 4 files) was downloaded from the Olympus platform of Great Learning.
To implement the techniques learned as a part of the course with the following learning outcomes:
# mounting the drive to use the files in the notebook
from google.colab import drive
drive.mount('/content/drive')
Drive already mounted at /content/drive; to attempt to forcibly remount, call drive.mount("/content/drive", force_remount=True).
# Import the needed libraries.
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import cv2
import os
import itertools
import seaborn as sns # For data visualization.
import tensorflow as tf
import keras
from sklearn.metrics import confusion_matrix # Confusion Matrix.
from sklearn.model_selection import train_test_split # Library for train, test, and split.
from keras.preprocessing import image # Image.
from keras.layers import Dense, Dropout, Flatten, Conv2D, MaxPool2D # NN Layers.
from keras.callbacks import EarlyStopping # Regularzation method to prevent the overfitting.
from keras.callbacks import ModelCheckpoint # Callback for Storing model amd weights.
from keras.preprocessing.image import ImageDataGenerator # Image Augumentation Libraries.
from keras.callbacks import ReduceLROnPlateau # Learning Rate callback.
from tensorflow.keras.optimizers import RMSprop,Adam # Optimiers for optimizing the model.
from tensorflow.keras.models import Sequential # Sequential API for sequential model.
from tensorflow.keras.layers import Dense, Dropout, Flatten # Importing the different layers.
from tensorflow.keras.layers import Conv2D, MaxPooling2D, BatchNormalization, Activation, Input, LeakyReLU,Activation
from tensorflow.keras.utils import to_categorical # To perform one-hot encoding.
from tensorflow.keras.models import Sequential, Model
from tensorflow.keras import losses, optimizers # Optimizers.
from tensorflow.keras.models import load_model # Loading the pre-trained model.
from IPython.display import display, HTML,Markdown
import warnings
warnings.filterwarnings("ignore", category=DeprecationWarning)
%%javascript
IPython.OutputArea.auto_scroll_threshold = 9999;
nptrainImages = np.load('trainimage.npy')
dftrainLabels = pd.read_csv("trainLabels.csv")
nptestImages = np.load('testimage.npy')
dftestLabels = pd.read_csv("testLabels.csv")
# Shape of the training image dataset.
nptrainImages.shape
(251, 128, 128, 3)
Observations:
# Sample of the training image dataset.
nptrainImages[0]
array([[[ 16, 16, 16],
[ 31, 31, 31],
[ 39, 39, 39],
...,
[ 27, 27, 27],
[ 51, 51, 51],
[ 83, 83, 83]],
[[ 23, 23, 23],
[ 21, 21, 21],
[ 41, 41, 41],
...,
[244, 244, 244],
[236, 236, 236],
[ 67, 67, 67]],
[[ 18, 18, 18],
[ 31, 31, 31],
[ 37, 37, 37],
...,
[ 80, 80, 80],
[ 69, 69, 69],
[ 58, 58, 58]],
...,
[[ 5, 5, 5],
[ 4, 4, 4],
[ 4, 4, 4],
...,
[ 0, 0, 0],
[ 0, 0, 0],
[ 4, 4, 4]],
[[ 1, 1, 1],
[ 2, 2, 2],
[ 2, 2, 2],
...,
[ 0, 0, 0],
[ 0, 0, 0],
[ 0, 0, 0]],
[[ 4, 4, 4],
[ 2, 2, 2],
[253, 253, 253],
...,
[ 0, 0, 0],
[ 0, 0, 0],
[ 0, 0, 0]]], dtype=uint8)
# Shape of the training labels dataset.
dftrainLabels.shape
(251, 1)
Observations:
# Shape of the test image dataset.
nptestImages.shape
(66, 128, 128, 3)
Observations:
# Sample of the test image dataset.
nptestImages[0]
array([[[ 86, 86, 86],
[109, 109, 109],
[104, 104, 104],
...,
[ 36, 36, 36],
[ 33, 33, 33],
[ 28, 28, 28]],
[[ 99, 99, 99],
[ 96, 96, 96],
[116, 116, 116],
...,
[ 31, 31, 31],
[ 29, 29, 29],
[ 28, 28, 28]],
[[105, 105, 105],
[105, 105, 105],
[117, 117, 117],
...,
[ 32, 32, 32],
[ 29, 29, 29],
[ 27, 27, 27]],
...,
[[ 56, 56, 56],
[ 8, 8, 8],
[ 5, 5, 5],
...,
[ 16, 16, 16],
[ 17, 17, 17],
[ 57, 57, 57]],
[[ 9, 9, 9],
[ 7, 7, 7],
[ 4, 4, 4],
...,
[ 17, 17, 17],
[ 17, 17, 17],
[ 17, 17, 17]],
[[ 9, 9, 9],
[ 8, 8, 8],
[ 4, 4, 4],
...,
[ 16, 16, 16],
[ 17, 17, 17],
[ 17, 17, 17]]], dtype=uint8)
# Shape of the test labels dataset.
dftestLabels.shape
(66, 1)
# X-ray labels random sample.
dftrainLabels.sample(n=10,random_state=1)
| Label | |
|---|---|
| 67 | Viral Pneumonia |
| 250 | Normal |
| 230 | Normal |
| 161 | Covid |
| 91 | Covid |
| 224 | Normal |
| 58 | Viral Pneumonia |
| 234 | Normal |
| 180 | Covid |
| 4 | Viral Pneumonia |
# Number of X-ray images for each each type of label.
lbl_val_counts= dftrainLabels.value_counts().to_frame('Count').reset_index()
lbl_val_counts
| Label | Count | |
|---|---|---|
| 0 | Covid | 111 |
| 1 | Viral Pneumonia | 70 |
| 2 | Normal | 70 |
Observations:
# Visualizing the data distribution with a pie chart.
plt.pie(lbl_val_counts['Count'], labels=lbl_val_counts['Label'], autopct='%1.0f%%',radius=2.5)
plt.show()
Observations:
dftrainLabels.isna().sum().to_frame('NaCount')
| NaCount | |
|---|---|
| Label | 0 |
Observation:
Are all images the same size?
# Examine the X-ray image dimensions. Collect all images which are different from the dimensions 128x128x3.
img_dim_data=[]
for i in range(nptrainImages.shape[0]):
if(str(nptrainImages[i].shape) != '(128, 128, 3)'):
#print(str(i)+lblData.iloc[0]['Label']+str(imgData[i].shape))
img_dim_data.append([i,dftrainLabels.iloc[0]['Label'],nptrainImages[i].shape])
dimension_df = pd.DataFrame(data=img_dim_data,columns=['img_num','Diagnosis','dimension'])
dimension_df.head()
| img_num | Diagnosis | dimension |
|---|
# Sample X-ray diagnosis index for data visualization.
row_data=[]
for c in lbl_val_counts['Label'].sort_values():
row_data.append([c,dftrainLabels[dftrainLabels['Label']==c].head(6).index.values])
sample_xray_index_df=pd.DataFrame(data=row_data,columns=['Diagnosis','Sample Indexes']).sort_values(by='Diagnosis')
sample_xray_index_df
| Diagnosis | Sample Indexes | |
|---|---|---|
| 0 | Covid | [70, 71, 72, 73, 74, 75] |
| 1 | Normal | [181, 182, 183, 184, 185, 186] |
| 2 | Viral Pneumonia | [0, 1, 2, 3, 4, 5] |
Observation:
# Helper function to print sample images.
def print_sample_images(imageData):
fig = plt.figure(figsize = (25,25))
i=0
for c in lbl_val_counts['Label'].sort_values():
indexVals = sample_xray_index_df[sample_xray_index_df['Diagnosis'] == c]['Sample Indexes'].values
for imgIndx in indexVals.all():
fp=imageData[imgIndx]
label=c
#print(i)
ax = fig.add_subplot(12, 6, i+1)
i=i+1
# to plot without rescaling, remove target_size
#fn = image.load_img(fp,color_mode='rgb')
plt.imshow(fp,cmap='gray')
plt.title(label)
plt.axis('off')
plt.tight_layout(w_pad=-25,h_pad=1)
plt.show()
# Converting color images to gray scale.
imgData_gray=np.empty(shape=(nptrainImages.shape[0],128,128))
for i in range(nptrainImages.shape[0]):
imgData_gray[i]=cv2.cvtColor(nptrainImages[i], cv2.COLOR_BGR2GRAY)
imgData_gray.shape
(251, 128, 128)
# Print a sample of the gray scale images.
print_sample_images(imgData_gray)
Observations:
print_sample_images(nptrainImages)
img_dict={}
for c in lbl_val_counts['Label']:
img_dict[c]=[]
for i in range(dftrainLabels.shape[0]):
label=dftrainLabels.iloc[i]['Label']
img_dict[label].append(nptrainImages[i])
# Helper function to print Avg/mean of an image per category/Label.
def print_images_mean():
fig = plt.figure(figsize = (45,45))
i=0
for c in lbl_val_counts['Label'].sort_values():
mean_img = np.mean(img_dict[c],axis =0)
mean_img = mean_img/np.amax(mean_img)
# reshape it back to a matrix
mean_img = mean_img.reshape((128,128,3))
#print(i)
ax = fig.add_subplot(12, 3, i+1)
i=i+1
# to plot without rescaling, remove target_size
#fn = image.load_img(fp,color_mode='rgb')
plt.imshow(mean_img)
plt.title(f'Average {c}')
plt.axis('off')
plt.tight_layout(w_pad=-250,h_pad=5)
plt.show()
print_images_mean()
Observations:
imgData_original=np.copy(nptrainImages,subok=True)
print_sample_images(imgData_original)
Observation:
Image blurring is achieved by convolving the image with a low-pass filter kernel useful for removing noise.
Using 5x5 Gaussian blurring kernal to reduce image noise and detail as a part of pre-processing.
imgData_blurred=np.copy(nptrainImages,subok=True)
for i in range(nptrainImages.shape[0]):
cv2.GaussianBlur(nptrainImages[i], (5, 5), 0)
print_sample_images(imgData_blurred)
Observation:
# Normalizing the pixel values.
# Original dataset (no Gaussian Blurring et al.).
imgData_original_normalized=np.copy(imgData_original,subok=True)
imgData_original_normalized=imgData_original/255.0
# Blurred dataset.
imgData_blurred_normalized=np.copy(imgData_blurred,subok=True)
imgData_blurred_normalized=imgData_blurred_normalized/255.0
# Image reshaping original dataset with no Gaussian Blurring et al..
imgData_original_normalized=imgData_original_normalized.reshape(-1,128,128,3)
# Sample image data after normalization (original dataset with no Gaussian Blurring et al.).
imgData_original_normalized[0]
array([[[0.0627451 , 0.0627451 , 0.0627451 ],
[0.12156863, 0.12156863, 0.12156863],
[0.15294118, 0.15294118, 0.15294118],
...,
[0.10588235, 0.10588235, 0.10588235],
[0.2 , 0.2 , 0.2 ],
[0.3254902 , 0.3254902 , 0.3254902 ]],
[[0.09019608, 0.09019608, 0.09019608],
[0.08235294, 0.08235294, 0.08235294],
[0.16078431, 0.16078431, 0.16078431],
...,
[0.95686275, 0.95686275, 0.95686275],
[0.9254902 , 0.9254902 , 0.9254902 ],
[0.2627451 , 0.2627451 , 0.2627451 ]],
[[0.07058824, 0.07058824, 0.07058824],
[0.12156863, 0.12156863, 0.12156863],
[0.14509804, 0.14509804, 0.14509804],
...,
[0.31372549, 0.31372549, 0.31372549],
[0.27058824, 0.27058824, 0.27058824],
[0.22745098, 0.22745098, 0.22745098]],
...,
[[0.01960784, 0.01960784, 0.01960784],
[0.01568627, 0.01568627, 0.01568627],
[0.01568627, 0.01568627, 0.01568627],
...,
[0. , 0. , 0. ],
[0. , 0. , 0. ],
[0.01568627, 0.01568627, 0.01568627]],
[[0.00392157, 0.00392157, 0.00392157],
[0.00784314, 0.00784314, 0.00784314],
[0.00784314, 0.00784314, 0.00784314],
...,
[0. , 0. , 0. ],
[0. , 0. , 0. ],
[0. , 0. , 0. ]],
[[0.01568627, 0.01568627, 0.01568627],
[0.00784314, 0.00784314, 0.00784314],
[0.99215686, 0.99215686, 0.99215686],
...,
[0. , 0. , 0. ],
[0. , 0. , 0. ],
[0. , 0. , 0. ]]])
# Image reshaping blurred dataset.
imgData_blurred_normalized=imgData_blurred_normalized.reshape(-1,128,128,3)
# Sample image of blurred data after normalization
imgData_blurred_normalized[0]
array([[[0.0627451 , 0.0627451 , 0.0627451 ],
[0.12156863, 0.12156863, 0.12156863],
[0.15294118, 0.15294118, 0.15294118],
...,
[0.10588235, 0.10588235, 0.10588235],
[0.2 , 0.2 , 0.2 ],
[0.3254902 , 0.3254902 , 0.3254902 ]],
[[0.09019608, 0.09019608, 0.09019608],
[0.08235294, 0.08235294, 0.08235294],
[0.16078431, 0.16078431, 0.16078431],
...,
[0.95686275, 0.95686275, 0.95686275],
[0.9254902 , 0.9254902 , 0.9254902 ],
[0.2627451 , 0.2627451 , 0.2627451 ]],
[[0.07058824, 0.07058824, 0.07058824],
[0.12156863, 0.12156863, 0.12156863],
[0.14509804, 0.14509804, 0.14509804],
...,
[0.31372549, 0.31372549, 0.31372549],
[0.27058824, 0.27058824, 0.27058824],
[0.22745098, 0.22745098, 0.22745098]],
...,
[[0.01960784, 0.01960784, 0.01960784],
[0.01568627, 0.01568627, 0.01568627],
[0.01568627, 0.01568627, 0.01568627],
...,
[0. , 0. , 0. ],
[0. , 0. , 0. ],
[0.01568627, 0.01568627, 0.01568627]],
[[0.00392157, 0.00392157, 0.00392157],
[0.00784314, 0.00784314, 0.00784314],
[0.00784314, 0.00784314, 0.00784314],
...,
[0. , 0. , 0. ],
[0. , 0. , 0. ],
[0. , 0. , 0. ]],
[[0.01568627, 0.01568627, 0.01568627],
[0.00784314, 0.00784314, 0.00784314],
[0.99215686, 0.99215686, 0.99215686],
...,
[0. , 0. , 0. ],
[0. , 0. , 0. ],
[0. , 0. , 0. ]]])
# Imagedata shape no blurring and normalized.
imgData_original_normalized.shape
(251, 128, 128, 3)
# Imagedata shape after blurred and normalized.
imgData_blurred_normalized.shape
(251, 128, 128, 3)
# Printing sample images no blurring and normalization pre processes.
print_sample_images(imgData_original_normalized)
# Printing sample images after blur and normalization pre processes.
print_sample_images(imgData_blurred_normalized)
Observations:
Since the backgrounds of the blurred images are clear solid black and not filled with lots of background objects there is no need to apply further masking pre-processing technoques to the X-ray images.
There is the letter "R" that appears in the foreground of some of the images but we will ignore those for the time being.
from sklearn.preprocessing import LabelBinarizer
enc_data=[]
enc = LabelBinarizer()
y_encoded = enc.fit_transform(dftrainLabels['Label']) # For training.
y_encoded[0]
array([0, 0, 1])
y_encoded_test = enc.fit_transform(dftestLabels['Label']) # For test.
y_encoded_test[0]
array([0, 0, 1])
# Get the unique label entries from dftrainLabels dataset.
lblData_no_dups= dftrainLabels.drop_duplicates()
lblData_no_dups=lblData_no_dups.reset_index().rename(columns={'index':'row_id'})
# Add encoded value from y_encoded based on row_id
lblData_no_dups['Encoded_val']=[y_encoded[x] for x in lblData_no_dups['row_id']]
# Add class label from y_encoded based on row_id
lblData_no_dups['Class_Label']=[np.argmax(y_encoded[x]) for x in lblData_no_dups['row_id']]
lblData_no_dups.drop(columns=['row_id'],inplace=True)
lblData_no_dups.sort_values(by='Class_Label',ascending=True)
| Label | Encoded_val | Class_Label | |
|---|---|---|---|
| 1 | Covid | [1, 0, 0] | 0 |
| 2 | Normal | [0, 1, 0] | 1 |
| 0 | Viral Pneumonia | [0, 0, 1] | 2 |
X_train_original = imgData_original_normalized
y_train_original = y_encoded
print(X_train_original.shape)
print(y_train_original.shape)
(251, 128, 128, 3) (251, 3)
X_test_original, X_val_original, y_test_original, y_val_original = train_test_split(nptestImages,y_encoded_test, test_size = 0.5, random_state=2)
print(X_test_original.shape)
print(X_val_original.shape)
print(y_test_original.shape)
print(y_val_original.shape)
(33, 128, 128, 3) (33, 128, 128, 3) (33, 3) (33, 3)
X_train_blurred = imgData_blurred_normalized
y_train_blurred = y_encoded
print(X_train_blurred.shape)
print(y_train_blurred.shape)
(251, 128, 128, 3) (251, 3)
X_test_blurred, X_val_blurred, y_test_blurred, y_val_blurred = train_test_split(nptestImages,y_encoded_test, test_size = 0.5, random_state=2)
print(X_test_blurred.shape)
print(X_val_blurred.shape)
print(y_test_blurred.shape)
print(y_val_blurred.shape)
(33, 128, 128, 3) (33, 128, 128, 3) (33, 3) (33, 3)
Create/train four models.
For all 4 models:
Models Details
Model1: Basic Model
Model2: Model with more layers and Dropout
Model3: Model with Dropout, DataAugumentaion, LRReduction Callbacks to address the overfitting.
# Prints the confusion matrix.
def print_confusion_matrix(model,X_values,y_values,title_val):
# Predict the values from the validation dataset.
Y_pred = model.predict(X_values)
# Convert predictions classes to one hot vectors.
Y_pred_classes = np.argmax(Y_pred,axis = 1)
# Convert validation observations to one hot vectors.
Y_true = np.argmax(y_values,axis = 1)
# Compute the confusion matrix.
confusion_mtx = confusion_matrix(Y_true, Y_pred_classes)
# Plot the confusion matrix.
plot_confusion_matrix(confusion_mtx, title=title_val, classes = range(3))
# Plots the evaluation metrics.
def plot_metrics(history):
# List of metrics to be plotted.
metrics = ['loss', 'accuracy', 'precision', 'recall']
fig = plt.figure(figsize = (10,10))
# For every metric in the above list plot diagram.
for n, metric in enumerate(metrics):
name = metric.replace("_"," ").capitalize()
plt.subplot(2,2,n+1)
plt.plot(history.epoch, history.history[metric], label='Train')
#Validation metric plot
plt.plot(history.epoch, history.history['val_'+metric],
label='Val')
plt.title('model '+metric)
plt.xlabel('Epoch')
plt.ylabel(name)
if metric == 'loss':
plt.ylim([0, plt.ylim()[1]])
elif metric == 'auc':
plt.ylim([0.8,1])
else:
plt.ylim([0,1])
plt.tight_layout()
plt.legend()
# This will collect misclassification info (like predicted and actual labels) in a dataframe and prints.
# Sample imaged for the misclassifications will be shown.
def print_visualize_misclassifications(model_val,X_test_value,y_test_value):
display_mainHeading("Model Misclassifications")
# Predicted label.
predicted_test = np.argmax(model_val.predict(X_test_value),axis = 1)
# Index of predicted label.
wrong_pred_indices=np.where(predicted_test != np.argmax(y_test_value,axis=1))[0]
# Loop through all misclassified row indexes and preparing data to add to dataframe.
wrong_pred_data=[]
for i in range(wrong_pred_indices.size):
wrong_row_indx=wrong_pred_indices[i]
wrong_pred_data.append([wrong_row_indx,enc.classes_[predicted_test[wrong_row_indx]]+'('+str(predicted_test[wrong_row_indx])+')',enc.classes_[np.argmax(y_test_value[wrong_row_indx])]+'('+str(np.argmax(y_test_value[wrong_row_indx]))+')'])
# Creating the misclassification dataframe.
wrong_pred_df = pd.DataFrame(data=wrong_pred_data,columns=['RowNo','Predicted Diagnosis (Label)','Actual Diagnosis (Label)'])
misclas_str='Total Number of misclassifications with this model = ' +str(wrong_pred_df.shape[0])+' . Following are some samples...'
display (Markdown('<font color="orange">'+misclas_str+'</font>'))
# Display('Total Number of misclassifications with this model =' +str(wrong_pred_df.shape[0])).
display(wrong_pred_df.head(10))
f, axarr = plt.subplots(1, 2, figsize=(10, 10))
cnt = (wrong_pred_df.shape[0] - 1)
# Shows the image of first sample from misclassification dataframe built.
display (Markdown('<BR>'))
display_subHeading("Model Misclassification Samples")
axarr[0].imshow(X_test_value[wrong_pred_df['RowNo'][0]])
axarr[0].set_title("Predicted = "+wrong_pred_df['Predicted Diagnosis (Label)'][0]+"\nActual = "+wrong_pred_df['Actual Diagnosis (Label)'][0])
#plt.imshow(X_test_value[wrong_pred_df['RowNo'][0]])
#plt.show()
#print("Predicted class for RowNo "+str(wrong_pred_df['RowNo'][0])+" : {}".format(wrong_pred_df['Predicted Seedling(Label)'][0]))
#print("Actual class for RowNo "+str(wrong_pred_df['RowNo'][0])+" : {}".format(wrong_pred_df['Actual Seedling(Label)'][0]))
#Shows the image of fifth sample from misclassification dataframe built
#display (Markdown('<BR>'))
#plt.imshow(X_test_value[wrong_pred_df['RowNo'][5]])
#print("Predicted class for RowNo "+str(wrong_pred_df['RowNo'][5])+" : {}".format(wrong_pred_df['Predicted Seedling(Label)'][5]))
#print("Actual class for Row No "+str(wrong_pred_df['RowNo'][5])+" : {}".format(wrong_pred_df['Actual Seedling(Label)'][5]))
axarr[1].imshow(X_test_value[wrong_pred_df['RowNo'][cnt]])
axarr[1].set_title("Predicted = "+wrong_pred_df['Predicted Diagnosis (Label)'][cnt]+"\nActual = "+wrong_pred_df['Actual Diagnosis (Label)'][cnt])
plt.show()
return wrong_pred_df
# Prepare empty performance metrics dataframe to capture for all models.
columns_of_interest=['Classifier Mdl Name','Trn Accuracy','Tst Accuracy','Val Accuracy','Trn Precision','Tst Precision','Val Precision','Trn Recall','Tst Recall','Val Recall']
model_perf_df = pd.DataFrame(data=[],columns=['Classifier Mdl Name','Trn Accuracy','Tst Accuracy','Val Accuracy','Trn Precision','Tst Precision','Val Precision','Trn Recall','Tst Recall','Val Recall','Mdl Config'])
# Plot the confusion matrix.
def plot_confusion_matrix(cm, classes,
normalize=False,
title='Confusion matrix',
cmap=plt.cm.Blues):
"""
This function prints and plots the confusion matrix.
Normalization can be applied by setting `normalize=True`.
"""
plt.subplots(figsize=(22,7)) # Set the size of the plot.
plt.imshow(cm, interpolation='nearest', cmap=cmap)
plt.title(title)
plt.colorbar()
tick_marks = np.arange(len(classes))
plt.xticks(tick_marks, classes, rotation=45)
plt.yticks(tick_marks, classes)
if normalize:
cm = cm.astype('float') / cm.sum(axis=1)[:, np.newaxis]
thresh = cm.max() / 2.
for i, j in itertools.product(range(cm.shape[0]), range(cm.shape[1])):
plt.text(j, i, cm[i, j],
horizontalalignment="center",
color="white" if cm[i, j] > thresh else "black")
plt.tight_layout()
plt.ylabel('True label')
plt.xlabel('Predicted label')
# Print all performace metrics and capture them in a dataframe for all models comparision.
def evaluate_and_store_perf_metrics(model,model_name,model_description,X_train_value, y_train_value,X_validation, y_validation,X_test_value, y_test_value):
#model_perf_df = pd.DataFrame(data=[],columns=['Classifier Mdl Name','Trn Accuracy','Tst Accuracy','Trn Precision','Tst Precision','Trn Recall','Tst Recall','Mdl Config'])
display_mainHeading("Model Evaluation Train/Test/Validation Scores")
# Validation data metrics.
val_scores = model.evaluate(X_validation, y_validation, verbose=0, batch_size = 64)
# Train data metrics.
train_scores = model.evaluate(X_train_value, y_train_value, verbose=0, batch_size = 64)
# Test data metrics.
test_scores = model.evaluate(X_test_value, y_test_value, verbose=0, batch_size = 64)
# Append all train/test/valdation scores to dataframe.
model_perf_df.loc[len(model_perf_df.index)] = [
model_name,
train_scores[1],
test_scores[1],
val_scores[1],
train_scores[2],
test_scores[2],
val_scores[2],
train_scores[3],
test_scores[3],
val_scores[3],
model.get_config()]
# Display(model_perf_df).
return model_perf_df
# Display helper for the start and end of the model run.
def display_mainHeading(string):
display (Markdown('<BR><center> <strong><font color="blue"><----------------------------------'+string+'-------------------------------------></font></strong></center><BR>'))
#Display helper for the sub headings within the model run.
def display_subHeading(string):
display (Markdown('<center> <strong><font color="green"><-------------'+string+'-------------></font></strong></center>'))
es = EarlyStopping(monitor='val_loss', mode='min', verbose=1, patience=20)
mc = ModelCheckpoint('model1_best_model.h5', monitor='val_accuracy', mode='max', verbose=1, save_best_only=True)
model1 = Sequential()
# Conv/Maxpool.
model1.add(Conv2D(filters = 16, kernel_size = (5,5),padding = 'Same',
activation ='relu', input_shape = (128,128,3)))
model1.add(MaxPool2D(pool_size=(2,2)))
# Conv/Maxpool.
model1.add(Conv2D(filters = 32, kernel_size = (3,3),padding = 'Same',
activation ='relu'))
model1.add(MaxPool2D(pool_size=(2,2)))
# Conv/Maxpool.
model1.add(Conv2D(filters = 64, kernel_size = (3,3),padding = 'Same',
activation ='relu'))
model1.add(MaxPool2D(pool_size=(2,2)))
# Flattening.
model1.add(Flatten())
# Dense layer.
model1.add(Dense(512, activation = "relu"))
model1.add(Dense(3, activation = "softmax"))
# Compile the model.
model1.compile(optimizer = "rmsprop" , loss = "categorical_crossentropy", metrics=["accuracy","Recall","Precision"])
model1.summary()
Model: "sequential" _________________________________________________________________ Layer (type) Output Shape Param # ================================================================= conv2d (Conv2D) (None, 128, 128, 16) 1216 _________________________________________________________________ max_pooling2d (MaxPooling2D) (None, 64, 64, 16) 0 _________________________________________________________________ conv2d_1 (Conv2D) (None, 64, 64, 32) 4640 _________________________________________________________________ max_pooling2d_1 (MaxPooling2 (None, 32, 32, 32) 0 _________________________________________________________________ conv2d_2 (Conv2D) (None, 32, 32, 64) 18496 _________________________________________________________________ max_pooling2d_2 (MaxPooling2 (None, 16, 16, 64) 0 _________________________________________________________________ flatten (Flatten) (None, 16384) 0 _________________________________________________________________ dense (Dense) (None, 512) 8389120 _________________________________________________________________ dense_1 (Dense) (None, 3) 1539 ================================================================= Total params: 8,415,011 Trainable params: 8,415,011 Non-trainable params: 0 _________________________________________________________________
# Train the model.
history1=model1.fit(X_train_blurred,
y_train_blurred, # It expects integers due to the sparse_categorical_crossentropy loss function.
epochs=50, # Number of iterations over the entire dataset to train on.
batch_size=64,validation_data = (X_val_blurred,y_val_blurred),callbacks=[es, mc],use_multiprocessing=True) # Number of samples per gradient update for training.
Epoch 1/50 4/4 [==============================] - 6s 1s/step - loss: 6.5669 - accuracy: 0.4223 - recall: 0.2988 - precision: 0.4870 - val_loss: 36.2221 - val_accuracy: 0.4242 - val_recall: 0.4242 - val_precision: 0.4242 Epoch 00001: val_accuracy improved from -inf to 0.42424, saving model to model1_best_model.h5 Epoch 2/50 4/4 [==============================] - 5s 1s/step - loss: 0.9318 - accuracy: 0.6016 - recall: 0.3386 - precision: 0.6538 - val_loss: 27.3527 - val_accuracy: 0.7273 - val_recall: 0.7273 - val_precision: 0.7273 Epoch 00002: val_accuracy improved from 0.42424 to 0.72727, saving model to model1_best_model.h5 Epoch 3/50 4/4 [==============================] - 5s 1s/step - loss: 0.7835 - accuracy: 0.7052 - recall: 0.6215 - precision: 0.7059 - val_loss: 193.4232 - val_accuracy: 0.3636 - val_recall: 0.3636 - val_precision: 0.3636 Epoch 00003: val_accuracy did not improve from 0.72727 Epoch 4/50 4/4 [==============================] - 5s 1s/step - loss: 0.7171 - accuracy: 0.6733 - recall: 0.6335 - precision: 0.7195 - val_loss: 97.1229 - val_accuracy: 0.6667 - val_recall: 0.6667 - val_precision: 0.6667 Epoch 00004: val_accuracy did not improve from 0.72727 Epoch 5/50 4/4 [==============================] - 5s 1s/step - loss: 0.3234 - accuracy: 0.8645 - recall: 0.8526 - precision: 0.8807 - val_loss: 300.5949 - val_accuracy: 0.6061 - val_recall: 0.6061 - val_precision: 0.6061 Epoch 00005: val_accuracy did not improve from 0.72727 Epoch 6/50 4/4 [==============================] - 5s 1s/step - loss: 0.4459 - accuracy: 0.8606 - recall: 0.8486 - precision: 0.8694 - val_loss: 76.8789 - val_accuracy: 0.7576 - val_recall: 0.7576 - val_precision: 0.7576 Epoch 00006: val_accuracy improved from 0.72727 to 0.75758, saving model to model1_best_model.h5 Epoch 7/50 4/4 [==============================] - 5s 1s/step - loss: 0.2062 - accuracy: 0.9203 - recall: 0.9124 - precision: 0.9385 - val_loss: 74.6539 - val_accuracy: 0.6970 - val_recall: 0.6970 - val_precision: 0.6970 Epoch 00007: val_accuracy did not improve from 0.75758 Epoch 8/50 4/4 [==============================] - 5s 1s/step - loss: 0.3010 - accuracy: 0.8765 - recall: 0.8606 - precision: 0.8816 - val_loss: 92.0889 - val_accuracy: 0.6667 - val_recall: 0.6667 - val_precision: 0.6667 Epoch 00008: val_accuracy did not improve from 0.75758 Epoch 9/50 4/4 [==============================] - 5s 1s/step - loss: 0.2145 - accuracy: 0.9363 - recall: 0.9363 - precision: 0.9438 - val_loss: 104.0727 - val_accuracy: 0.6667 - val_recall: 0.6667 - val_precision: 0.6667 Epoch 00009: val_accuracy did not improve from 0.75758 Epoch 10/50 4/4 [==============================] - 5s 1s/step - loss: 0.2239 - accuracy: 0.9402 - recall: 0.9283 - precision: 0.9433 - val_loss: 41.5434 - val_accuracy: 0.8788 - val_recall: 0.8788 - val_precision: 0.8788 Epoch 00010: val_accuracy improved from 0.75758 to 0.87879, saving model to model1_best_model.h5 Epoch 11/50 4/4 [==============================] - 5s 1s/step - loss: 0.1962 - accuracy: 0.9323 - recall: 0.9243 - precision: 0.9355 - val_loss: 41.4072 - val_accuracy: 0.7879 - val_recall: 0.7879 - val_precision: 0.7879 Epoch 00011: val_accuracy did not improve from 0.87879 Epoch 12/50 4/4 [==============================] - 5s 1s/step - loss: 0.0812 - accuracy: 0.9801 - recall: 0.9801 - precision: 0.9840 - val_loss: 35.1982 - val_accuracy: 0.9091 - val_recall: 0.9091 - val_precision: 0.9091 Epoch 00012: val_accuracy improved from 0.87879 to 0.90909, saving model to model1_best_model.h5 Epoch 13/50 4/4 [==============================] - 5s 1s/step - loss: 0.0836 - accuracy: 0.9721 - recall: 0.9721 - precision: 0.9799 - val_loss: 77.7801 - val_accuracy: 0.8182 - val_recall: 0.8182 - val_precision: 0.8182 Epoch 00013: val_accuracy did not improve from 0.90909 Epoch 14/50 4/4 [==============================] - 5s 1s/step - loss: 0.0890 - accuracy: 0.9602 - recall: 0.9602 - precision: 0.9602 - val_loss: 60.8982 - val_accuracy: 0.7879 - val_recall: 0.7879 - val_precision: 0.7879 Epoch 00014: val_accuracy did not improve from 0.90909 Epoch 15/50 4/4 [==============================] - 5s 1s/step - loss: 0.1512 - accuracy: 0.9641 - recall: 0.9562 - precision: 0.9639 - val_loss: 50.6376 - val_accuracy: 0.8788 - val_recall: 0.8788 - val_precision: 0.8788 Epoch 00015: val_accuracy did not improve from 0.90909 Epoch 16/50 4/4 [==============================] - 5s 1s/step - loss: 0.0291 - accuracy: 0.9920 - recall: 0.9920 - precision: 0.9960 - val_loss: 63.6351 - val_accuracy: 0.8182 - val_recall: 0.8182 - val_precision: 0.8182 Epoch 00016: val_accuracy did not improve from 0.90909 Epoch 17/50 4/4 [==============================] - 5s 1s/step - loss: 0.3074 - accuracy: 0.8606 - recall: 0.8566 - precision: 0.8704 - val_loss: 314.6042 - val_accuracy: 0.5455 - val_recall: 0.5455 - val_precision: 0.5455 Epoch 00017: val_accuracy did not improve from 0.90909 Epoch 18/50 4/4 [==============================] - 5s 1s/step - loss: 0.2648 - accuracy: 0.9044 - recall: 0.8924 - precision: 0.9180 - val_loss: 64.1339 - val_accuracy: 0.8182 - val_recall: 0.8182 - val_precision: 0.8182 Epoch 00018: val_accuracy did not improve from 0.90909 Epoch 19/50 4/4 [==============================] - 5s 1s/step - loss: 0.5546 - accuracy: 0.8526 - recall: 0.8406 - precision: 0.8543 - val_loss: 186.3378 - val_accuracy: 0.6667 - val_recall: 0.6667 - val_precision: 0.6667 Epoch 00019: val_accuracy did not improve from 0.90909 Epoch 20/50 4/4 [==============================] - 5s 1s/step - loss: 0.1339 - accuracy: 0.9641 - recall: 0.9562 - precision: 0.9677 - val_loss: 65.7208 - val_accuracy: 0.8485 - val_recall: 0.8485 - val_precision: 0.8485 Epoch 00020: val_accuracy did not improve from 0.90909 Epoch 21/50 4/4 [==============================] - 5s 1s/step - loss: 0.0518 - accuracy: 0.9880 - recall: 0.9880 - precision: 0.9920 - val_loss: 72.4929 - val_accuracy: 0.8788 - val_recall: 0.8788 - val_precision: 0.8788 Epoch 00021: val_accuracy did not improve from 0.90909 Epoch 22/50 4/4 [==============================] - 5s 1s/step - loss: 0.0269 - accuracy: 0.9920 - recall: 0.9880 - precision: 0.9920 - val_loss: 129.2077 - val_accuracy: 0.7879 - val_recall: 0.7879 - val_precision: 0.7879 Epoch 00022: val_accuracy did not improve from 0.90909 Epoch 00022: early stopping
plot_metrics(history1)
print_confusion_matrix(model1,X_val_blurred,y_val_blurred,"Confusion Matrix for Validation data")
print_confusion_matrix(model1,X_test_blurred,y_test_blurred,'Confusion Matrix for Test data')
misclass_df1 = print_visualize_misclassifications(model1,X_test_blurred,y_test_blurred)
Total Number of misclassifications with this model = 4 . Following are some samples...
| RowNo | Predicted Diagnosis (Label) | Actual Diagnosis (Label) | |
|---|---|---|---|
| 0 | 4 | Covid(0) | Viral Pneumonia(2) |
| 1 | 7 | Covid(0) | Normal(1) |
| 2 | 11 | Covid(0) | Viral Pneumonia(2) |
| 3 | 29 | Covid(0) | Viral Pneumonia(2) |
evaluate_and_store_perf_metrics(model1,'Model1','Basic Model',X_train_blurred,y_train_blurred,X_val_blurred,y_val_blurred,X_test_blurred,y_test_blurred)
| Classifier Mdl Name | Trn Accuracy | Tst Accuracy | Val Accuracy | Trn Precision | Tst Precision | Val Precision | Trn Recall | Tst Recall | Val Recall | Mdl Config | |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | Model1 | 0.996016 | 0.878788 | 0.787879 | 0.996016 | 0.878788 | 0.787879 | 0.996016 | 0.878788 | 0.787879 | {'name': 'sequential', 'layers': [{'class_name... |
es = EarlyStopping(monitor='val_loss', mode='min', verbose=1, patience=10)
mc = ModelCheckpoint('model2_best_model.h5', monitor='val_accuracy', mode='max', verbose=1, save_best_only=True)
model2 = Sequential()
# Conv/Maxpool/Dropout.
model2.add(Conv2D(filters = 64, kernel_size = (5,5),padding = 'Same',
activation ='relu', input_shape = (128,128,3)))
model2.add(MaxPool2D(pool_size=(2,2)))
model2.add(Dropout(0.25))
# Conv/Maxpool/Dropout.
model2.add(Conv2D(filters = 128, kernel_size = (3,3),padding = 'Same',
activation ='relu'))
model2.add(MaxPool2D(pool_size=(2,2), strides=(2,2)))
model2.add(Dropout(0.25))
# Conv/Maxpool/Dropout.
model2.add(Conv2D(filters = 128, kernel_size = (3,3),padding = 'Same',
activation ='relu'))
model2.add(MaxPool2D(pool_size=(2,2), strides=(2,2)))
model2.add(Dropout(0.3))
# Conv/Maxpool/Dropout.
model2.add(Conv2D(filters = 128, kernel_size = (2,2),padding = 'Same',
activation ='relu'))
model2.add(MaxPool2D(pool_size=(2,2), strides=(2,2)))
model2.add(Dropout(0.3))
# Conv/Maxpool/Dropout.
model2.add(Conv2D(filters = 256, kernel_size = (2,2),padding = 'Same',
activation ='relu'))
model2.add(MaxPool2D(pool_size=(2,2), strides=(2,2)))
model2.add(Dropout(0.3))
# Flattening.
model2.add(Flatten())
# Dense/Dropot.
model2.add(Dense(1024, activation = "relu"))
model2.add(Dropout(0.5))
# Output layer.
model2.add(Dense(3, activation = "softmax"))
# Optimizer.
optimizer = Adam(learning_rate=0.001, beta_1=0.9, beta_2=0.999)
# Compile the model.
model2.compile(optimizer = optimizer , loss = "categorical_crossentropy", metrics=["accuracy","Recall","Precision"])
model2.summary()
Model: "sequential_1" _________________________________________________________________ Layer (type) Output Shape Param # ================================================================= conv2d_3 (Conv2D) (None, 128, 128, 64) 4864 _________________________________________________________________ max_pooling2d_3 (MaxPooling2 (None, 64, 64, 64) 0 _________________________________________________________________ dropout (Dropout) (None, 64, 64, 64) 0 _________________________________________________________________ conv2d_4 (Conv2D) (None, 64, 64, 128) 73856 _________________________________________________________________ max_pooling2d_4 (MaxPooling2 (None, 32, 32, 128) 0 _________________________________________________________________ dropout_1 (Dropout) (None, 32, 32, 128) 0 _________________________________________________________________ conv2d_5 (Conv2D) (None, 32, 32, 128) 147584 _________________________________________________________________ max_pooling2d_5 (MaxPooling2 (None, 16, 16, 128) 0 _________________________________________________________________ dropout_2 (Dropout) (None, 16, 16, 128) 0 _________________________________________________________________ conv2d_6 (Conv2D) (None, 16, 16, 128) 65664 _________________________________________________________________ max_pooling2d_6 (MaxPooling2 (None, 8, 8, 128) 0 _________________________________________________________________ dropout_3 (Dropout) (None, 8, 8, 128) 0 _________________________________________________________________ conv2d_7 (Conv2D) (None, 8, 8, 256) 131328 _________________________________________________________________ max_pooling2d_7 (MaxPooling2 (None, 4, 4, 256) 0 _________________________________________________________________ dropout_4 (Dropout) (None, 4, 4, 256) 0 _________________________________________________________________ flatten_1 (Flatten) (None, 4096) 0 _________________________________________________________________ dense_2 (Dense) (None, 1024) 4195328 _________________________________________________________________ dropout_5 (Dropout) (None, 1024) 0 _________________________________________________________________ dense_3 (Dense) (None, 3) 3075 ================================================================= Total params: 4,621,699 Trainable params: 4,621,699 Non-trainable params: 0 _________________________________________________________________
#Train the model
history2=model2.fit(X_train_blurred,
y_train_blurred, #It expects integers because of the sparse_categorical_crossentropy loss function
epochs=50, #number of iterations over the entire dataset to train on
batch_size=64,validation_data = (X_val_blurred,y_val_blurred),callbacks=[es, mc],use_multiprocessing=True)#number of samples per gradient update for training
Epoch 1/50 4/4 [==============================] - 21s 5s/step - loss: 1.6587 - accuracy: 0.3705 - recall: 0.1594 - precision: 0.4878 - val_loss: 4.1207 - val_accuracy: 0.3636 - val_recall: 0.3636 - val_precision: 0.3636 Epoch 00001: val_accuracy improved from -inf to 0.36364, saving model to model2_best_model.h5 Epoch 2/50 4/4 [==============================] - 19s 5s/step - loss: 1.1190 - accuracy: 0.3546 - recall: 0.0000e+00 - precision: 0.0000e+00 - val_loss: 1.5186 - val_accuracy: 0.3636 - val_recall: 0.3636 - val_precision: 0.3636 Epoch 00002: val_accuracy did not improve from 0.36364 Epoch 3/50 4/4 [==============================] - 19s 5s/step - loss: 1.0928 - accuracy: 0.4661 - recall: 0.0000e+00 - precision: 0.0000e+00 - val_loss: 1.3698 - val_accuracy: 0.3030 - val_recall: 0.2727 - val_precision: 0.2812 Epoch 00003: val_accuracy did not improve from 0.36364 Epoch 4/50 4/4 [==============================] - 19s 5s/step - loss: 1.0821 - accuracy: 0.4422 - recall: 0.0000e+00 - precision: 0.0000e+00 - val_loss: 15.7067 - val_accuracy: 0.3030 - val_recall: 0.3030 - val_precision: 0.3030 Epoch 00004: val_accuracy did not improve from 0.36364 Epoch 5/50 4/4 [==============================] - 19s 5s/step - loss: 1.0527 - accuracy: 0.4422 - recall: 0.0956 - precision: 0.7500 - val_loss: 35.5603 - val_accuracy: 0.3030 - val_recall: 0.3030 - val_precision: 0.3030 Epoch 00005: val_accuracy did not improve from 0.36364 Epoch 6/50 4/4 [==============================] - 19s 5s/step - loss: 1.0176 - accuracy: 0.4582 - recall: 0.3227 - precision: 0.5664 - val_loss: 13.3497 - val_accuracy: 0.3636 - val_recall: 0.3636 - val_precision: 0.3636 Epoch 00006: val_accuracy did not improve from 0.36364 Epoch 7/50 4/4 [==============================] - 19s 5s/step - loss: 0.8533 - accuracy: 0.6853 - recall: 0.2988 - precision: 0.7732 - val_loss: 113.4447 - val_accuracy: 0.3636 - val_recall: 0.3636 - val_precision: 0.3636 Epoch 00007: val_accuracy did not improve from 0.36364 Epoch 8/50 4/4 [==============================] - 19s 5s/step - loss: 0.8249 - accuracy: 0.6574 - recall: 0.6215 - precision: 0.6753 - val_loss: 162.5038 - val_accuracy: 0.3636 - val_recall: 0.3636 - val_precision: 0.3636 Epoch 00008: val_accuracy did not improve from 0.36364 Epoch 9/50 4/4 [==============================] - 19s 5s/step - loss: 0.7114 - accuracy: 0.7490 - recall: 0.6295 - precision: 0.7596 - val_loss: 14.2642 - val_accuracy: 0.6970 - val_recall: 0.6970 - val_precision: 0.6970 Epoch 00009: val_accuracy improved from 0.36364 to 0.69697, saving model to model2_best_model.h5 Epoch 10/50 4/4 [==============================] - 19s 5s/step - loss: 0.6577 - accuracy: 0.7610 - recall: 0.6175 - precision: 0.8115 - val_loss: 87.6480 - val_accuracy: 0.4848 - val_recall: 0.4848 - val_precision: 0.4848 Epoch 00010: val_accuracy did not improve from 0.69697 Epoch 11/50 4/4 [==============================] - 19s 5s/step - loss: 0.5009 - accuracy: 0.7809 - recall: 0.7490 - precision: 0.8210 - val_loss: 68.2915 - val_accuracy: 0.6667 - val_recall: 0.6667 - val_precision: 0.6667 Epoch 00011: val_accuracy did not improve from 0.69697 Epoch 12/50 4/4 [==============================] - 19s 5s/step - loss: 0.5560 - accuracy: 0.7410 - recall: 0.7171 - precision: 0.7725 - val_loss: 27.8530 - val_accuracy: 0.6061 - val_recall: 0.6061 - val_precision: 0.6061 Epoch 00012: val_accuracy did not improve from 0.69697 Epoch 13/50 4/4 [==============================] - 19s 5s/step - loss: 0.5275 - accuracy: 0.7849 - recall: 0.7410 - precision: 0.8455 - val_loss: 127.0427 - val_accuracy: 0.5758 - val_recall: 0.5758 - val_precision: 0.5758 Epoch 00013: val_accuracy did not improve from 0.69697 Epoch 00013: early stopping
plot_metrics(history2)
print_confusion_matrix(model2,X_val_blurred,y_val_blurred,"Confusion Matrix for Validation data")
print_confusion_matrix(model2,X_test_blurred,y_test_blurred,'Confusion Matrix for Test data')
misclass_df2 = print_visualize_misclassifications(model2,X_test_blurred,y_test_blurred)
Total Number of misclassifications with this model = 11 . Following are some samples...
| RowNo | Predicted Diagnosis (Label) | Actual Diagnosis (Label) | |
|---|---|---|---|
| 0 | 0 | Viral Pneumonia(2) | Normal(1) |
| 1 | 4 | Covid(0) | Viral Pneumonia(2) |
| 2 | 7 | Covid(0) | Normal(1) |
| 3 | 9 | Covid(0) | Viral Pneumonia(2) |
| 4 | 11 | Covid(0) | Viral Pneumonia(2) |
| 5 | 16 | Viral Pneumonia(2) | Normal(1) |
| 6 | 19 | Covid(0) | Normal(1) |
| 7 | 21 | Viral Pneumonia(2) | Normal(1) |
| 8 | 24 | Covid(0) | Normal(1) |
| 9 | 29 | Covid(0) | Viral Pneumonia(2) |
evaluate_and_store_perf_metrics(model2,'Model2','Model with dropout and maxpool',X_train_blurred,y_train_blurred,X_val_blurred,y_val_blurred,X_test_blurred,y_test_blurred)
| Classifier Mdl Name | Trn Accuracy | Tst Accuracy | Val Accuracy | Trn Precision | Tst Precision | Val Precision | Trn Recall | Tst Recall | Val Recall | Mdl Config | |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | Model1 | 0.996016 | 0.878788 | 0.787879 | 0.996016 | 0.878788 | 0.787879 | 0.996016 | 0.878788 | 0.787879 | {'name': 'sequential', 'layers': [{'class_name... |
| 1 | Model2 | 0.741036 | 0.666667 | 0.575758 | 0.629482 | 0.666667 | 0.575758 | 0.818653 | 0.666667 | 0.575758 | {'name': 'sequential_1', 'layers': [{'class_na... |
# Data Augumentation.
datagen = ImageDataGenerator(
featurewise_center=False, # Set input mean to 0 over the dataset.
samplewise_center=False, # Set each sample mean to 0.
featurewise_std_normalization=False, # Divide inputs by std of the dataset.
samplewise_std_normalization=False, # Divide each input by its std.
zca_whitening=False, # Apply ZCA whitening.
rotation_range=10, # Randomly rotate images in the range (degrees, 0 to 180).
zoom_range = 0.1, # Randomly zoom image.
width_shift_range=0.1, # Randomly shift images horizontally (fraction of total width).
height_shift_range=0.1, # Randomly shift images vertically (fraction of total height).
horizontal_flip=False, # Randomly flip images.
vertical_flip=False) # Randomly flip images.
datagen.fit(X_train_blurred)
# Learning rate reduction callback.
learning_rate_reduction = ReduceLROnPlateau(monitor='val_accuracy',
patience=3,
verbose=1,
factor=0.5,
min_lr=0.00001)
es = EarlyStopping(monitor='val_loss', mode='min', verbose=1, patience=8)
batch_size = 38
mc = ModelCheckpoint('model3_best_model.h5', monitor='val_accuracy', mode='max', verbose=1, save_best_only=True)
model3 = Sequential()
#Conv/Maxpool/Dropout
model3.add(Conv2D(filters = 64, kernel_size = (5,5),padding = 'Same',
activation ='relu', input_shape = (128,128,3)))
model3.add(MaxPool2D(pool_size=(2,2)))
model3.add(Dropout(0.25))
#Conv/Maxpool/Dropout
model3.add(Conv2D(filters = 128, kernel_size = (3,3),padding = 'Same',
activation ='relu'))
model3.add(MaxPool2D(pool_size=(2,2), strides=(2,2)))
model3.add(Dropout(0.25))
#Conv/Maxpool/Dropout
model3.add(Conv2D(filters = 128, kernel_size = (3,3),padding = 'Same',
activation ='relu'))
model3.add(MaxPool2D(pool_size=(2,2), strides=(2,2)))
model3.add(Dropout(0.3))
#Conv/Maxpool/Dropout
model3.add(Conv2D(filters = 128, kernel_size = (2,2),padding = 'Same',
activation ='relu'))
model3.add(MaxPool2D(pool_size=(2,2), strides=(2,2)))
model3.add(Dropout(0.3))
#Conv/Maxpool/Dropout
model3.add(Conv2D(filters = 256, kernel_size = (2,2),padding = 'Same',
activation ='relu'))
model3.add(MaxPool2D(pool_size=(2,2), strides=(2,2)))
model3.add(Dropout(0.3))
#Flattening
model3.add(Flatten())
#Dense/Dropout
model3.add(Dense(1024, activation = "relu"))
model3.add(Dropout(0.5))
#output layer
model3.add(Dense(3, activation = "softmax"))
#Optimizer
optimizer = Adam(learning_rate=0.001, beta_1=0.9, beta_2=0.999)
#Compile the model
model3.compile(optimizer = optimizer , loss = "categorical_crossentropy", metrics=["accuracy","Recall","Precision"])
model3.summary()
Model: "sequential_2" _________________________________________________________________ Layer (type) Output Shape Param # ================================================================= conv2d_8 (Conv2D) (None, 128, 128, 64) 4864 _________________________________________________________________ max_pooling2d_8 (MaxPooling2 (None, 64, 64, 64) 0 _________________________________________________________________ dropout_6 (Dropout) (None, 64, 64, 64) 0 _________________________________________________________________ conv2d_9 (Conv2D) (None, 64, 64, 128) 73856 _________________________________________________________________ max_pooling2d_9 (MaxPooling2 (None, 32, 32, 128) 0 _________________________________________________________________ dropout_7 (Dropout) (None, 32, 32, 128) 0 _________________________________________________________________ conv2d_10 (Conv2D) (None, 32, 32, 128) 147584 _________________________________________________________________ max_pooling2d_10 (MaxPooling (None, 16, 16, 128) 0 _________________________________________________________________ dropout_8 (Dropout) (None, 16, 16, 128) 0 _________________________________________________________________ conv2d_11 (Conv2D) (None, 16, 16, 128) 65664 _________________________________________________________________ max_pooling2d_11 (MaxPooling (None, 8, 8, 128) 0 _________________________________________________________________ dropout_9 (Dropout) (None, 8, 8, 128) 0 _________________________________________________________________ conv2d_12 (Conv2D) (None, 8, 8, 256) 131328 _________________________________________________________________ max_pooling2d_12 (MaxPooling (None, 4, 4, 256) 0 _________________________________________________________________ dropout_10 (Dropout) (None, 4, 4, 256) 0 _________________________________________________________________ flatten_2 (Flatten) (None, 4096) 0 _________________________________________________________________ dense_4 (Dense) (None, 1024) 4195328 _________________________________________________________________ dropout_11 (Dropout) (None, 1024) 0 _________________________________________________________________ dense_5 (Dense) (None, 3) 3075 ================================================================= Total params: 4,621,699 Trainable params: 4,621,699 Non-trainable params: 0 _________________________________________________________________
# Train the model.
history3 = model3.fit(datagen.flow(X_train_blurred,y_train_blurred, batch_size=batch_size),
epochs = 50, validation_data = (X_val_blurred,y_val_blurred),
verbose = 2, steps_per_epoch=X_train_blurred.shape[0] // batch_size
, callbacks=[mc,es,learning_rate_reduction])
Epoch 1/50 6/6 - 19s - loss: 1.2818 - accuracy: 0.3146 - recall: 0.1080 - precision: 0.4340 - val_loss: 2.7898 - val_accuracy: 0.3636 - val_recall: 0.3636 - val_precision: 0.3636 Epoch 00001: val_accuracy improved from -inf to 0.36364, saving model to model3_best_model.h5 Epoch 2/50 6/6 - 17s - loss: 1.0985 - accuracy: 0.3286 - recall: 0.0000e+00 - precision: 0.0000e+00 - val_loss: 1.5315 - val_accuracy: 0.3333 - val_recall: 0.3333 - val_precision: 0.3333 Epoch 00002: val_accuracy did not improve from 0.36364 Epoch 3/50 6/6 - 17s - loss: 1.0806 - accuracy: 0.4601 - recall: 0.0000e+00 - precision: 0.0000e+00 - val_loss: 32.2196 - val_accuracy: 0.3030 - val_recall: 0.3030 - val_precision: 0.3030 Epoch 00003: val_accuracy did not improve from 0.36364 Epoch 4/50 6/6 - 17s - loss: 1.0361 - accuracy: 0.4366 - recall: 0.1127 - precision: 0.7500 - val_loss: 76.0736 - val_accuracy: 0.3030 - val_recall: 0.3030 - val_precision: 0.3030 Epoch 00004: val_accuracy did not improve from 0.36364 Epoch 00004: ReduceLROnPlateau reducing learning rate to 0.0005000000237487257. Epoch 5/50 6/6 - 18s - loss: 0.9242 - accuracy: 0.5526 - recall: 0.3991 - precision: 0.6842 - val_loss: 90.7724 - val_accuracy: 0.3636 - val_recall: 0.3636 - val_precision: 0.3636 Epoch 00005: val_accuracy did not improve from 0.36364 Epoch 6/50 6/6 - 17s - loss: 0.9147 - accuracy: 0.5915 - recall: 0.4554 - precision: 0.6424 - val_loss: 109.6255 - val_accuracy: 0.3030 - val_recall: 0.3030 - val_precision: 0.3030 Epoch 00006: val_accuracy did not improve from 0.36364 Epoch 7/50 6/6 - 17s - loss: 0.7576 - accuracy: 0.6901 - recall: 0.5728 - precision: 0.7771 - val_loss: 67.3606 - val_accuracy: 0.4545 - val_recall: 0.4545 - val_precision: 0.4545 Epoch 00007: val_accuracy improved from 0.36364 to 0.45455, saving model to model3_best_model.h5 Epoch 8/50 6/6 - 17s - loss: 0.7794 - accuracy: 0.6667 - recall: 0.6056 - precision: 0.7207 - val_loss: 63.2724 - val_accuracy: 0.5152 - val_recall: 0.5152 - val_precision: 0.5152 Epoch 00008: val_accuracy improved from 0.45455 to 0.51515, saving model to model3_best_model.h5 Epoch 9/50 6/6 - 18s - loss: 0.6838 - accuracy: 0.6754 - recall: 0.5877 - precision: 0.7128 - val_loss: 75.3230 - val_accuracy: 0.4545 - val_recall: 0.4545 - val_precision: 0.4545 Epoch 00009: val_accuracy did not improve from 0.51515 Epoch 10/50 6/6 - 17s - loss: 0.5682 - accuracy: 0.7887 - recall: 0.7042 - precision: 0.8333 - val_loss: 81.4709 - val_accuracy: 0.6061 - val_recall: 0.6061 - val_precision: 0.6061 Epoch 00010: val_accuracy improved from 0.51515 to 0.60606, saving model to model3_best_model.h5 Epoch 00010: early stopping
plot_metrics(history3)
print_confusion_matrix(model3,X_val_blurred,y_val_blurred,"Confusion Matrix for Validation data")
print_confusion_matrix(model3,X_test_blurred,y_test_blurred,"Confusion Matrix for Validation data")
misclass_df = print_visualize_misclassifications(model3,X_test_blurred,y_test_blurred)
Total Number of misclassifications with this model = 9 . Following are some samples...
| RowNo | Predicted Diagnosis (Label) | Actual Diagnosis (Label) | |
|---|---|---|---|
| 0 | 0 | Covid(0) | Normal(1) |
| 1 | 4 | Covid(0) | Viral Pneumonia(2) |
| 2 | 9 | Covid(0) | Viral Pneumonia(2) |
| 3 | 11 | Covid(0) | Viral Pneumonia(2) |
| 4 | 12 | Normal(1) | Covid(0) |
| 5 | 16 | Covid(0) | Normal(1) |
| 6 | 24 | Covid(0) | Normal(1) |
| 7 | 29 | Covid(0) | Viral Pneumonia(2) |
| 8 | 31 | Covid(0) | Viral Pneumonia(2) |
evaluate_and_store_perf_metrics(model3,"Model3","Model with Data Augumentation and LR Plataeu",X_train_blurred,y_train_blurred,X_val_blurred,y_val_blurred,X_test_blurred,y_test_blurred)
| Classifier Mdl Name | Trn Accuracy | Tst Accuracy | Val Accuracy | Trn Precision | Tst Precision | Val Precision | Trn Recall | Tst Recall | Val Recall | Mdl Config | |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | Model1 | 0.996016 | 0.878788 | 0.787879 | 0.996016 | 0.878788 | 0.787879 | 0.996016 | 0.878788 | 0.787879 | {'name': 'sequential', 'layers': [{'class_name... |
| 1 | Model2 | 0.741036 | 0.666667 | 0.575758 | 0.629482 | 0.666667 | 0.575758 | 0.818653 | 0.666667 | 0.575758 | {'name': 'sequential_1', 'layers': [{'class_na... |
| 2 | Model3 | 0.733068 | 0.727273 | 0.606061 | 0.442231 | 0.727273 | 0.606061 | 0.810219 | 0.727273 | 0.606061 | {'name': 'sequential_2', 'layers': [{'class_na... |
es = EarlyStopping(monitor='val_loss', mode='min', verbose=1, patience=20)
mc = ModelCheckpoint('model4_best_model.h5', monitor='val_accuracy', mode='max', verbose=1, save_best_only=True)
model4 = Sequential()
# Conv/Maxpool.
model4.add(Conv2D(filters = 16, kernel_size = (5,5),padding = 'Same',
activation ='relu', input_shape = (128,128,3)))
model4.add(MaxPool2D(pool_size=(2,2)))
# Conv/Maxpool.
model4.add(Conv2D(filters = 32, kernel_size = (3,3),padding = 'Same',
activation ='relu'))
model4.add(MaxPool2D(pool_size=(2,2)))
# Conv/Maxpool.
model4.add(Conv2D(filters = 64, kernel_size = (3,3),padding = 'Same',
activation ='relu'))
model4.add(MaxPool2D(pool_size=(2,2)))
# Flattening.
model4.add(Flatten())
# Dense layer.
model4.add(Dense(512, activation = "relu"))
model4.add(Dense(3, activation = "softmax"))
# Compile the model.
model4.compile(optimizer = "rmsprop" , loss = "categorical_crossentropy", metrics=["accuracy","Recall","Precision"])
model4.summary()
Model: "sequential_3" _________________________________________________________________ Layer (type) Output Shape Param # ================================================================= conv2d_13 (Conv2D) (None, 128, 128, 16) 1216 _________________________________________________________________ max_pooling2d_13 (MaxPooling (None, 64, 64, 16) 0 _________________________________________________________________ conv2d_14 (Conv2D) (None, 64, 64, 32) 4640 _________________________________________________________________ max_pooling2d_14 (MaxPooling (None, 32, 32, 32) 0 _________________________________________________________________ conv2d_15 (Conv2D) (None, 32, 32, 64) 18496 _________________________________________________________________ max_pooling2d_15 (MaxPooling (None, 16, 16, 64) 0 _________________________________________________________________ flatten_3 (Flatten) (None, 16384) 0 _________________________________________________________________ dense_6 (Dense) (None, 512) 8389120 _________________________________________________________________ dense_7 (Dense) (None, 3) 1539 ================================================================= Total params: 8,415,011 Trainable params: 8,415,011 Non-trainable params: 0 _________________________________________________________________
# Train the model.
history4=model4.fit(X_train_original,
y_train_original, # It expects integers due to the sparse_categorical_crossentropy loss function.
epochs=50, # Number of iterations over the entire dataset to train on.
batch_size=64,validation_data = (X_val_original,y_val_original),callbacks=[es, mc],use_multiprocessing=True) # Number of samples per gradient update for training.
Epoch 1/50 4/4 [==============================] - 6s 1s/step - loss: 3.1935 - accuracy: 0.4024 - recall: 0.2829 - precision: 0.3966 - val_loss: 137.3161 - val_accuracy: 0.3636 - val_recall: 0.3636 - val_precision: 0.3636 Epoch 00001: val_accuracy improved from -inf to 0.36364, saving model to model4_best_model.h5 Epoch 2/50 4/4 [==============================] - 5s 1s/step - loss: 0.9756 - accuracy: 0.5378 - recall: 0.1673 - precision: 0.6667 - val_loss: 8.6354 - val_accuracy: 0.8182 - val_recall: 0.8182 - val_precision: 0.8182 Epoch 00002: val_accuracy improved from 0.36364 to 0.81818, saving model to model4_best_model.h5 Epoch 3/50 4/4 [==============================] - 5s 1s/step - loss: 0.5939 - accuracy: 0.7928 - recall: 0.6653 - precision: 0.8698 - val_loss: 32.4208 - val_accuracy: 0.6667 - val_recall: 0.6667 - val_precision: 0.6667 Epoch 00003: val_accuracy did not improve from 0.81818 Epoch 4/50 4/4 [==============================] - 5s 1s/step - loss: 0.3667 - accuracy: 0.8924 - recall: 0.8127 - precision: 0.9273 - val_loss: 58.6031 - val_accuracy: 0.6970 - val_recall: 0.6970 - val_precision: 0.6970 Epoch 00004: val_accuracy did not improve from 0.81818 Epoch 5/50 4/4 [==============================] - 5s 1s/step - loss: 0.3631 - accuracy: 0.8486 - recall: 0.8367 - precision: 0.8750 - val_loss: 57.4205 - val_accuracy: 0.6970 - val_recall: 0.6970 - val_precision: 0.6970 Epoch 00005: val_accuracy did not improve from 0.81818 Epoch 6/50 4/4 [==============================] - 5s 1s/step - loss: 0.4124 - accuracy: 0.8367 - recall: 0.8207 - precision: 0.8443 - val_loss: 18.1169 - val_accuracy: 0.7576 - val_recall: 0.7576 - val_precision: 0.7576 Epoch 00006: val_accuracy did not improve from 0.81818 Epoch 7/50 4/4 [==============================] - 5s 1s/step - loss: 0.4145 - accuracy: 0.8406 - recall: 0.8207 - precision: 0.8512 - val_loss: 28.0111 - val_accuracy: 0.8788 - val_recall: 0.8788 - val_precision: 0.8788 Epoch 00007: val_accuracy improved from 0.81818 to 0.87879, saving model to model4_best_model.h5 Epoch 8/50 4/4 [==============================] - 5s 1s/step - loss: 0.1950 - accuracy: 0.9363 - recall: 0.9203 - precision: 0.9506 - val_loss: 40.4153 - val_accuracy: 0.7879 - val_recall: 0.7879 - val_precision: 0.7879 Epoch 00008: val_accuracy did not improve from 0.87879 Epoch 9/50 4/4 [==============================] - 5s 1s/step - loss: 0.1652 - accuracy: 0.9283 - recall: 0.9203 - precision: 0.9277 - val_loss: 59.2918 - val_accuracy: 0.7273 - val_recall: 0.7273 - val_precision: 0.7273 Epoch 00009: val_accuracy did not improve from 0.87879 Epoch 10/50 4/4 [==============================] - 5s 1s/step - loss: 0.1608 - accuracy: 0.9402 - recall: 0.9363 - precision: 0.9476 - val_loss: 276.6011 - val_accuracy: 0.6364 - val_recall: 0.6364 - val_precision: 0.6364 Epoch 00010: val_accuracy did not improve from 0.87879 Epoch 11/50 4/4 [==============================] - 5s 1s/step - loss: 0.2061 - accuracy: 0.9124 - recall: 0.9124 - precision: 0.9160 - val_loss: 36.8831 - val_accuracy: 0.8788 - val_recall: 0.8788 - val_precision: 0.8788 Epoch 00011: val_accuracy did not improve from 0.87879 Epoch 12/50 4/4 [==============================] - 5s 1s/step - loss: 0.0857 - accuracy: 0.9721 - recall: 0.9681 - precision: 0.9720 - val_loss: 36.2102 - val_accuracy: 0.8788 - val_recall: 0.8788 - val_precision: 0.8788 Epoch 00012: val_accuracy did not improve from 0.87879 Epoch 13/50 4/4 [==============================] - 5s 1s/step - loss: 0.0348 - accuracy: 0.9960 - recall: 0.9960 - precision: 0.9960 - val_loss: 19.0665 - val_accuracy: 0.8788 - val_recall: 0.8788 - val_precision: 0.8788 Epoch 00013: val_accuracy did not improve from 0.87879 Epoch 14/50 4/4 [==============================] - 5s 1s/step - loss: 0.1672 - accuracy: 0.9562 - recall: 0.9562 - precision: 0.9639 - val_loss: 38.8619 - val_accuracy: 0.7879 - val_recall: 0.7879 - val_precision: 0.7879 Epoch 00014: val_accuracy did not improve from 0.87879 Epoch 15/50 4/4 [==============================] - 5s 1s/step - loss: 0.0398 - accuracy: 0.9960 - recall: 0.9960 - precision: 0.9960 - val_loss: 18.8376 - val_accuracy: 0.9091 - val_recall: 0.9091 - val_precision: 0.9091 Epoch 00015: val_accuracy improved from 0.87879 to 0.90909, saving model to model4_best_model.h5 Epoch 16/50 4/4 [==============================] - 5s 1s/step - loss: 0.0124 - accuracy: 1.0000 - recall: 1.0000 - precision: 1.0000 - val_loss: 34.7203 - val_accuracy: 0.8788 - val_recall: 0.8788 - val_precision: 0.8788 Epoch 00016: val_accuracy did not improve from 0.90909 Epoch 17/50 4/4 [==============================] - 5s 1s/step - loss: 0.0092 - accuracy: 0.9960 - recall: 0.9960 - precision: 0.9960 - val_loss: 390.4663 - val_accuracy: 0.6667 - val_recall: 0.6667 - val_precision: 0.6667 Epoch 00017: val_accuracy did not improve from 0.90909 Epoch 18/50 4/4 [==============================] - 5s 1s/step - loss: 0.2969 - accuracy: 0.8924 - recall: 0.8845 - precision: 0.9024 - val_loss: 12.3035 - val_accuracy: 0.8485 - val_recall: 0.8485 - val_precision: 0.8485 Epoch 00018: val_accuracy did not improve from 0.90909 Epoch 19/50 4/4 [==============================] - 5s 1s/step - loss: 0.0462 - accuracy: 1.0000 - recall: 0.9960 - precision: 1.0000 - val_loss: 18.7473 - val_accuracy: 0.8485 - val_recall: 0.8485 - val_precision: 0.8485 Epoch 00019: val_accuracy did not improve from 0.90909 Epoch 20/50 4/4 [==============================] - 5s 1s/step - loss: 0.0223 - accuracy: 0.9960 - recall: 0.9960 - precision: 0.9960 - val_loss: 76.6574 - val_accuracy: 0.8788 - val_recall: 0.8788 - val_precision: 0.8788 Epoch 00020: val_accuracy did not improve from 0.90909 Epoch 21/50 4/4 [==============================] - 5s 1s/step - loss: 0.0065 - accuracy: 1.0000 - recall: 1.0000 - precision: 1.0000 - val_loss: 23.4883 - val_accuracy: 0.8788 - val_recall: 0.8788 - val_precision: 0.8788 Epoch 00021: val_accuracy did not improve from 0.90909 Epoch 22/50 4/4 [==============================] - 5s 1s/step - loss: 0.0017 - accuracy: 1.0000 - recall: 1.0000 - precision: 1.0000 - val_loss: 30.4993 - val_accuracy: 0.8788 - val_recall: 0.8788 - val_precision: 0.8788 Epoch 00022: val_accuracy did not improve from 0.90909 Epoch 00022: early stopping
plot_metrics(history4)
print_confusion_matrix(model4,X_val_original,y_val_original,"Confusion Matrix for Validation data")
print_confusion_matrix(model4,X_test_original,y_test_original,'Confusion Matrix for Test data')
misclass_df4 = print_visualize_misclassifications(model4,X_test_original,y_test_original)
Total Number of misclassifications with this model = 4 . Following are some samples...
| RowNo | Predicted Diagnosis (Label) | Actual Diagnosis (Label) | |
|---|---|---|---|
| 0 | 3 | Viral Pneumonia(2) | Covid(0) |
| 1 | 4 | Covid(0) | Viral Pneumonia(2) |
| 2 | 24 | Viral Pneumonia(2) | Normal(1) |
| 3 | 29 | Normal(1) | Viral Pneumonia(2) |
evaluate_and_store_perf_metrics(model4,'Model4','Basic Model (Original Dataset - No Gaussian Blurring et al.)',X_train_original,y_train_original,X_val_original,y_val_original,X_test_original,y_test_original)
| Classifier Mdl Name | Trn Accuracy | Tst Accuracy | Val Accuracy | Trn Precision | Tst Precision | Val Precision | Trn Recall | Tst Recall | Val Recall | Mdl Config | |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | Model1 | 0.996016 | 0.878788 | 0.787879 | 0.996016 | 0.878788 | 0.787879 | 0.996016 | 0.878788 | 0.787879 | {'name': 'sequential', 'layers': [{'class_name... |
| 1 | Model2 | 0.741036 | 0.666667 | 0.575758 | 0.629482 | 0.666667 | 0.575758 | 0.818653 | 0.666667 | 0.575758 | {'name': 'sequential_1', 'layers': [{'class_na... |
| 2 | Model3 | 0.733068 | 0.727273 | 0.606061 | 0.442231 | 0.727273 | 0.606061 | 0.810219 | 0.727273 | 0.606061 | {'name': 'sequential_2', 'layers': [{'class_na... |
| 3 | Model4 | 1.000000 | 0.878788 | 0.878788 | 1.000000 | 0.878788 | 0.878788 | 1.000000 | 0.878788 | 0.878788 | {'name': 'sequential_3', 'layers': [{'class_na... |
pwd
'/content'
ls
drive/ model3_best_model.h5 testimage.npy trainLabels.csv model1_best_model.h5 model4_best_model.h5 testLabels.csv model2_best_model.h5 sample_data/ trainimage.npy
def test_model_blurred(pretrained_model,X_val_value,y_val_value):
y_actual=np.argmax(y_test_blurred[0])
test_image = np.expand_dims(X_val_value, axis = 0)
Y_pred=pretrained_model.predict(test_image)
y_pred = np.argmax(Y_pred, axis=1)
actual_str="The actual class = "+str(y_actual)+" and Class Label = "+enc.classes_[y_actual]
pred_str= "The predicted class = "+str(y_pred[0])+" and Class Label = "+enc.classes_[y_pred][0]
if(y_actual == y_pred ):
display (Markdown('<font color="green">'+actual_str+'</font>'))
display (Markdown('<font color="green">'+pred_str+'</font>'))
else:
display (Markdown('<font color="red">'+actual_str+'</font>'))
display (Markdown('<font color="red">'+pred_str+'</font>'))
pretrained_model1 = load_model('./model1_best_model.h5')
test_model_blurred(pretrained_model1,X_test_blurred[2],y_test_blurred[2])
The actual class = 1 and Class Label = Normal
The predicted class = 0 and Class Label = Covid
test_model_blurred(pretrained_model1,X_test_blurred[3],y_test_blurred[3])
The actual class = 1 and Class Label = Normal
The predicted class = 0 and Class Label = Covid
test_model_blurred(pretrained_model1,X_test_blurred[8],y_test_blurred[8])
The actual class = 1 and Class Label = Normal
The predicted class = 1 and Class Label = Normal
test_model_blurred(pretrained_model1,X_test_blurred[13],y_test_blurred[13])
The actual class = 1 and Class Label = Normal
The predicted class = 0 and Class Label = Covid
test_model_blurred(pretrained_model1,X_test_blurred[25],y_test_blurred[25])
The actual class = 1 and Class Label = Normal
The predicted class = 2 and Class Label = Viral Pneumonia
pretrained_model2 = load_model('./model2_best_model.h5')
test_model_blurred(pretrained_model2,X_test_blurred[2],y_test_blurred[2])
The actual class = 1 and Class Label = Normal
The predicted class = 0 and Class Label = Covid
test_model_blurred(pretrained_model2,X_test_blurred[3],y_test_blurred[3])
The actual class = 1 and Class Label = Normal
The predicted class = 2 and Class Label = Viral Pneumonia
test_model_blurred(pretrained_model2,X_test_blurred[8],y_test_blurred[8])
The actual class = 1 and Class Label = Normal
The predicted class = 1 and Class Label = Normal
test_model_blurred(pretrained_model2,X_test_blurred[13],y_test_blurred[13])
The actual class = 1 and Class Label = Normal
The predicted class = 0 and Class Label = Covid
test_model_blurred(pretrained_model2,X_test_blurred[25],y_test_blurred[25])
The actual class = 1 and Class Label = Normal
The predicted class = 2 and Class Label = Viral Pneumonia
pretrained_model3 = load_model('./model3_best_model.h5')
test_model_blurred(pretrained_model3,X_test_blurred[2],y_test_blurred[2])
The actual class = 1 and Class Label = Normal
The predicted class = 0 and Class Label = Covid
test_model_blurred(pretrained_model3,X_test_blurred[3],y_test_blurred[3])
The actual class = 1 and Class Label = Normal
The predicted class = 0 and Class Label = Covid
test_model_blurred(pretrained_model3,X_test_blurred[8],y_test_blurred[8])
The actual class = 1 and Class Label = Normal
The predicted class = 1 and Class Label = Normal
test_model_blurred(pretrained_model3,X_test_blurred[13],y_test_blurred[13])
The actual class = 1 and Class Label = Normal
The predicted class = 0 and Class Label = Covid
test_model_blurred(pretrained_model3,X_test_blurred[25],y_test_blurred[25])
The actual class = 1 and Class Label = Normal
The predicted class = 2 and Class Label = Viral Pneumonia
def test_model_original(pretrained_model,X_val_value,y_val_value):
y_actual=np.argmax(y_test_original[0])
test_image = np.expand_dims(X_val_value, axis = 0)
Y_pred=pretrained_model.predict(test_image)
y_pred = np.argmax(Y_pred, axis=1)
actual_str="The actual class = "+str(y_actual)+" and Class Label = "+enc.classes_[y_actual]
pred_str= "The predicted class = "+str(y_pred[0])+" and Class Label = "+enc.classes_[y_pred][0]
if(y_actual == y_pred ):
display (Markdown('<font color="green">'+actual_str+'</font>'))
display (Markdown('<font color="green">'+pred_str+'</font>'))
else:
display (Markdown('<font color="red">'+actual_str+'</font>'))
display (Markdown('<font color="red">'+pred_str+'</font>'))
pretrained_model4 = load_model('./model4_best_model.h5')
test_model_original(pretrained_model4,X_test_original[2],y_test_original[2])
The actual class = 1 and Class Label = Normal
The predicted class = 0 and Class Label = Covid
test_model_original(pretrained_model4,X_test_original[2],y_test_original[3])
The actual class = 1 and Class Label = Normal
The predicted class = 0 and Class Label = Covid
test_model_original(pretrained_model4,X_test_original[2],y_test_original[8])
The actual class = 1 and Class Label = Normal
The predicted class = 0 and Class Label = Covid
test_model_original(pretrained_model4,X_test_original[2],y_test_original[13])
The actual class = 1 and Class Label = Normal
The predicted class = 0 and Class Label = Covid
test_model_original(pretrained_model4,X_test_original[2],y_test_original[25])
The actual class = 1 and Class Label = Normal
The predicted class = 0 and Class Label = Covid
Observations:
model_perf_df
| Classifier Mdl Name | Trn Accuracy | Tst Accuracy | Val Accuracy | Trn Precision | Tst Precision | Val Precision | Trn Recall | Tst Recall | Val Recall | Mdl Config | |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | Model1 | 0.996016 | 0.878788 | 0.787879 | 0.996016 | 0.878788 | 0.787879 | 0.996016 | 0.878788 | 0.787879 | {'name': 'sequential', 'layers': [{'class_name... |
| 1 | Model2 | 0.741036 | 0.666667 | 0.575758 | 0.629482 | 0.666667 | 0.575758 | 0.818653 | 0.666667 | 0.575758 | {'name': 'sequential_1', 'layers': [{'class_na... |
| 2 | Model3 | 0.733068 | 0.727273 | 0.606061 | 0.442231 | 0.727273 | 0.606061 | 0.810219 | 0.727273 | 0.606061 | {'name': 'sequential_2', 'layers': [{'class_na... |
| 3 | Model4 | 1.000000 | 0.878788 | 0.878788 | 1.000000 | 0.878788 | 0.878788 | 1.000000 | 0.878788 | 0.878788 | {'name': 'sequential_3', 'layers': [{'class_na... |
Model1 (with Gaussian Blurring et al.) and Model4 (with no filtering techniques) both had good performance metrics approximately 90% and above in all three environments - train, validation, and test.
Model1 demonstrated less variety in misclassification errors with false positives of COVID19 when the diagnoses were actually Normal or Viral Pneumonia in 4 of the images.
A larger dataset of images would further improve the performance of the COVID19 classification.
Need to look at options to use pre-trained models using Transfer learning Techniques.
The most common misclassification overall in all 4 models was falsely predicting the diagnosis as COVID19 when in fact the diagnosis was Viral Pneumonia; this was followed by erroneosly predicting COVID19 when in fact the diagnisis was Normal.
Using Average/Mean of the images does not seem to be helpful overall although the differences in the 3 classifications does look visually distinct and striking to the human eye.
Model1 can be fine tuned further to reduce the misclassifications.